This paper is about Optimization.

Optimization is used to solve hard problems in industry such as minimizing the costs of manufacturing a product (e.g. amount of raw material needed for a particular order size), finding the optimal parameters for a manufactured good (e.g. building a car that meets safety regulations but also minimizes fuel consumption), allocating staff to teams so that a critical set of competencies is present in the final allocations (e.g. aircrews for a fleet of planes from a fixed pool of diversely skilled staff), and workflow scheduling (e.g. assigning tasks on a production line to minimize time to complete finished goods).

Optimization techniques are also used very widely in statistical inference and machine learning, e.g. to learn the weights of a complex predictive model (such as a neural network) or to approximate the maximum likelihood or maximum posterior probability solution to a non-convex parameter estimation problem.

This paper is very hands-on, with a substantial coding task nearly every week. Each basic algorithm I ask you to implement will typically need fewer than (say) 100 lines of code: The substantial nature comes from getting it to work sufficiently well for the problem at hand, which until you build some experience unfortunately is something of a dark art. You can use any programming language you are familiar with.

You are expected to get your hands dirty cutting your own code and to also participate enthusiastically in describing your successes (and failures) in applying various approaches to a carefully-curated selection of difficult optimization problems to your peers in class. Mutual support in getting working demo code implemented, as well as friendly competition to find the best-working variants are both whole-heartedly encouraged. For many real-world optimization problems, success comes from experimentation grounded in experience - the in-class "post mortems" are designed to encourage reflection on your own experience and to facilitate sharing of learned best practices. Moreover they are the main form of continuous assessment for this paper. I will moderate these sessions and from time-to-time may contribute some observations and suggestions based on my own experience and what I have learned from others.

This paper is mostly workshop-led, and it is anticipated that most learning will take place through hands-on experience from working on particular assigned problems and from sharing experiences in the workshops.

After some foundational lectures we will look at various different optimization algorithms - one a week - and some kinds of problems they are a good match for. For many of these algorithms pseudocode (or for others pointers to some libraries) will be provided, which students then implement (or use) and apply to some example problems provided by the lecturer.

Each week there will also be a workshop where we will look in detail at our experiences using the approach described in the previous week on the designated example problems. What worked well (and why)? What didn't? What else did we learn from our experience? These workshops will be run by students, to a format the lecturer will prescribe and moderate.

Final assessment will be in the form of a short reflective report: A set of notes, with discussion and personal insights, on the techniques and principles learned or discovered - the idea is that essentially students will write their own short text/cookbook based around the paper.

Lectures will be recorded and made available on Moodle shortly after they take place. I will hold physical f2f and virtual zoom office hours weekly at a regular time slot TBC.

If you are unable to attend lectures and Workshops in person this semester, please let me know as soon as possible. I may be able to provide some additional online contact time if necessary.

I estimate an average of 5-6 hours a week will need to be spent working on the weekly assignments for this paper. Expect some weeks to be easier than others (= more time than 6 hours some weeks, less than 5 hours in others). In general combinatorial problems tend to require more work than real-valued ones.

Allow about 30 hours in total to write your report. It will almost certainly take longer than you anticipate, so I strongly urge you to work on your first draft as you go along, e.g. by typing up your notes and observations. Latex is an excellent tool for doing this if you are familiar with it.